Universal Approximation Abilities of a Modular Differentiable Neural Network.

Approximation ability is one of the most important topics in the field of neural networks (NNs). Feedforward NNs, activated by rectified linear units and some of their specific smoothed versions, provide universal approximators to convex as well as continuous functions. However, most of these networks are investigated empirically, or their characteristics are analyzed based on specific operation rules. Moreover, an adequate level of interpretability of the networks is missing as well. In this work, we propose a class of new network architecture, built with reusable neural modules (functional blocks), to supply differentiable and interpretable approximators for convex and continuous target functions. Specifically, first, we introduce a concrete model construction mechanism with particular blocks based on differentiable programming and the composition essence of the max operator, extending the scope of existing activation functions. Moreover, explicit block diagrams are provided for a clear understanding of the external architecture and the internal processing mechanism. Subsequently, the approximation behavior of the proposed network to convex functions and continuous functions is rigorously proved as well, by virtue of mathematical induction. Finally, plenty of numerical experiments are conducted on a wide variety of problems, which exhibit the effectiveness and the superiority of the proposed model over some existing ones.

Metabolic profiling of aortic stenosis and hypertrophic cardiomyopathy identifies mechanistic contrasts in substrate utilization.

Aortic stenosis (AS) and hypertrophic cardiomyopathy (HCM) are distinct disorders leading to left ventricular hypertrophy (LVH), but whether cardiac metabolism substantially differs between these in humans remains to be elucidated. We undertook an invasive (aortic root, coronary sinus) metabolic profiling in patients with severe AS and HCM in comparison with non-LVH controls to investigate cardiac fuel selection and metabolic remodeling. These patients were assessed under different physiological states (at rest, during stress induced by pacing). The identified changes in the metabolome were further validated by metabolomic and orthogonal transcriptomic analysis, in separately recruited patient cohorts. We identified a highly discriminant metabolomic signature in severe AS in all samples, regardless of sampling site, characterized by striking accumulation of long-chain acylcarnitines, intermediates of fatty acid transport across the inner mitochondrial membrane, and validated this in a separate cohort. Mechanistically, we identify a downregulation in the PPAR-α transcriptional network, including expression of genes regulating fatty acid oxidation (FAO). In silico modeling of ß-oxidation demonstrated that flux could be inhibited by both the accumulation of fatty acids as a substrate for mitochondria and the accumulation of medium-chain carnitines which induce competitive inhibition of the acyl-CoA dehydrogenases. We present a comprehensive analysis of changes in the metabolic pathways (transcriptome to metabolome) in severe AS, and its comparison to HCM. Our results demonstrate a progressive impairment of ß-oxidation from HCM to AS, particularly for FAO of long-chain fatty acids, and that the PPAR-α signaling network may be a specific metabolic therapeutic target in AS.

Chaos, periodic structures, and multistability: Complex dynamical behaviors of an eco-epidemiological model in parameter planes.

The environmental characteristics of a biological system are imbibed in some particular parameters of that system. Significant changes in any system parameter exert influence on the system dynamics as well as the persistence of interacting species. In this article, we explore the rich and tangled dynamics of an eco-epidemiological system by studying different parametric planes of the system. In the parameter planes, we find a variety of complex and subtle properties of the system, like the presence of a variety of intricate regular structures within irregular regimes, that cannot be found through a single parameter variation. Also, we find a new type of structure like an "eye" in a parametric plane. We notice the bistability between distinct pairs of attractors and also identify the coexistence of three periodic attractors. The most notable observation of this study is the coexistence of three periodic attractors and a chaotic attractor, which is a rare occurrence in biological systems. We also plot the basins for each set of coexisting attractors and see the existence of fractal basins in the system, which look like a "conch." The appearance of fractal basins in a system causes enormous complications in predicting the system's state in the long run. Variations in initial conditions and changes in parameters in parametric planes are key to managing the behavior of a system.

Supervised Feature Selection via Collaborative Neurodynamic Optimization.

As a crucial part of machine learning and pattern recognition, feature selection aims at selecting a subset of the most informative features from the set of all available features. In this article, supervised feature selection is at first formulated as a mixed-integer optimization problem with an objective function of weighted feature redundancy and relevancy subject to a cardinality constraint on the number of selected features. It is equivalently reformulated as a bound-constrained mixed-integer optimization problem by augmenting the objective function with a penalty function for realizing the cardinality constraint. With additional bilinear and linear equality constraints for realizing the integrality constraints, it is further reformulated as a bound-constrained biconvex optimization problem with two more penalty terms. Two collaborative neurodynamic optimization (CNO) approaches are proposed for solving the formulated and reformulated feature selection problems. One of the proposed CNO approaches uses a population of discrete-time recurrent neural networks (RNNs), and the other use a pair of continuous-time projection networks operating concurrently on two timescales. Experimental results on 13 benchmark datasets are elaborated to substantiate the superiority of the CNO approaches to several mainstream methods in terms of average classification accuracy with three commonly used classifiers.

Bifurcation, chaos, multistability, and organized structures in a predator-prey model with vigilance.

There is not a single species that does not strive for survival. Every species has crafted specialized techniques to avoid possible dangers that mostly come from the side of their predators. Survival instincts in nature led prey populations to develop many anti-predator strategies. Vigilance is a well-observed effective antipredator strategy that influences predator-prey dynamics significantly. We consider a simple discrete-time predator-prey model assuming that vigilance affects the predation rate and the growth rate of the prey. We investigate the system dynamics by constructing isoperiodic and Lyapunov exponent diagrams with the simultaneous variation of the prey's growth rate and the strength of vigilance. We observe a series of different types of organized periodic structures with different kinds of period-adding phenomena. The usual period-bubbling phenomenon is shown near a shrimp-shaped periodic structure. We observe the presence of double and triple heterogeneous attractors. We also notice Wada basin boundaries in the system, which is quite rare in ecological systems. The complex dynamics of the system in biparameter space are explored through extensive numerical simulations.

Fractional Approximation of Broad Learning System.

Approximation ability is of much importance for neural networks. The broad learning system (BLS) (Chen and Liu, 2018), widely used in the industry with good performance, has been proved to be a universal approximator from the aspect of density. This kind of approximation property is very important, which proves the existence of the desired network but does not provide a means of construction that is commonly implemented through complexity aspect. Thus, such an approach lacks the advantage of determining constructively the network architecture and its weights. To the best of our knowledge, for a BLS, there is a few theory providing a constructive approach to obtain the network structure along with weights ensuring the approximation properties. By virtue of the long-term memory and nonlocality properties, fractional calculus has observed many distinctive applications. The purpose of this article is to study the BLS approximation ability constructively, which is valid for fractional case as well. Specifically, first we introduce two simplified BLSs by means of extending functions. For each of the simplified BLSs, an upper bound of error is derived through the modulus of continuity of Caputo fractional derivatives. As a result, two types of fractional convergent behaviors of BLS, that is: 1) pointwise and 2) uniform convergence, have been rigorously proved as well. Finally, some numerical experiments are conducted to demonstrate the approximation capabilities of BLSs.

Unsupervised feature selection via adaptive autoencoder with redundancy control.

Unsupervised feature selection is one of the efficient approaches to reduce the dimension of unlabeled high-dimensional data. We present a novel adaptive autoencoder with redundancy control (AARC) as an unsupervised feature selector. By adding two Group Lasso penalties to the objective function, AARC integrates unsupervised feature selection and determination of a compact network structure into a single framework. Besides, a penalty based on a measure of dependency between features (such as Pearson correlation, mutual information) is added to the objective function for controlling the level of redundancy in the selected features. To realize the desired effects of different regularizers in different phases of the training, we introduce adaptive parameters which change with iterations. In addition, a smoothing function is utilized to approximate the three penalties since they are not differentiable at the origin. An ablation study is carried out to validate the capabilities of redundancy control and structure optimization of AARC. Subsequently, comparisons with nine state-of-the-art methods illustrate the efficiency of AARC for unsupervised feature selection.

Atrial nitroso-redox balance and refractoriness following on-pump cardiac surgery: a randomized trial of atorvastatin.

AIMS: Systemic inflammation and increased activity of atrial NOX2-containing NADPH oxidases have been associated with the new onset of atrial fibrillation (AF) after cardiac surgery. In addition to lowering LDL-cholesterol, statins exert rapid anti-inflammatory and antioxidant effects, the clinical significance of which remains controversial. METHODS AND RESULTS: We first assessed the impact of cardiac surgery and cardiopulmonary bypass (CPB) on atrial nitroso-redox balance by measuring NO synthase (NOS) and GTP cyclohydrolase-1 (GCH-1) activity, biopterin content, and superoxide production in paired samples of the right atrial appendage obtained before (PRE) and after CPB and reperfusion (POST) in 116 patients. The effect of perioperative treatment with atorvastatin (80 mg once daily) on these parameters, blood biomarkers, and the post-operative atrial effective refractory period (AERP) was then evaluated in a randomized, double-blind, placebo-controlled study in 80 patients undergoing cardiac surgery on CPB. CPB and reperfusion led to a significant increase in atrial superoxide production (74% CI 71-76%, n = 46 paired samples, P < 0.0001) and a reduction in atrial tetrahydrobiopterin (BH4) (34% CI 33-35%, n = 36 paired samples, P < 0.01), and in GCH-1 (56% CI 55-58%, n = 26 paired samples, P < 0.001) and NOS activity (58% CI 52-67%, n = 20 paired samples, P < 0.001). Perioperative atorvastatin treatment prevented the effect of CPB and reperfusion on all parameters but had no significant effect on the postoperative right AERP, troponin release, or NT-proBNP after cardiac surgery. CONCLUSION: Perioperative statin therapy prevents post-reperfusion atrial nitroso-redox imbalance in patients undergoing on-pump cardiac surgery but has no significant impact on postoperative atrial refractoriness, perioperative myocardial injury, or markers of postoperative LV function. CLINICAL TRIAL REGISTRATION: https://clinicaltrials.gov/ct2/show/NCT01780740.

Exploring Covert States of Brain Dynamics via Fuzzy Inference Encoding.

Human brain inherently exhibits latent mental processes which are likely to change rapidly over time. A framework that adopts a fuzzy inference system is proposed to model the dynamics of the human brain. The fuzzy inference system is used to encode real-world data to represent the salient features of the EEG signals. Then, an unsupervised clustering is conducted on the extracted feature space to identify the brain (external and covert) states that respond to different cognitive demands. To understand the human state change, a state transition diagram is introduced, allowing visualization of connectivity patterns between every pair of states. We compute the transition probability between every pair of states to represent the relationships between the states. This state transition diagram is named as the Fuzzy Covert State Transition Diagram (FCOSTD), which helps the understanding of human states and human performance. We then apply FCOSTD on distracted driving experiments. FCOSTD successfully discovers the external and covert states, faithfully reveals the transition of the brain between states, and the route of the state change when humans are distracted during a driving task. The experimental results demonstrate that different subjects have similar states and inter-state transition behaviour (establishing the consistency of the system) but different ways to allocate brain resources as different actions are being taken.

Dynamics of a stage-structured predator-prey model: cost and benefit of fear-induced group defense.

In the predator-prey system, predators can affect the prey population (1) by direct killing and (2) by inducing predation fear, which ultimately force preys to adopt some anti-predator strategies. However, the anti-predator strategy is not the same for all individual preys of different life stages. Also, anti-predator behavior has both cost and benefit, but most of the mathematical models observed the dynamics by incorporating its cost only. In the present study, we formulate a predator-prey model dividing the prey population into two stages: juvenile and adult. We assume that adult preys are only adapting group defense as an anti-predator strategy when they are sensitive to predation. Group defense plays a positive role for adult prey by reducing their predation, but, on the negative side, it simultaneously decreases their reproductive potential. A parameter, anti-predator sensitivity is introduced to interlink both the benefit and cost of group defense. Our result shows that when adult preys are not showing anti-predator behavior, with an increase of maturation rate, the system exhibits a population cycle of abruptly increasing amplitude, which may drive all species of the system to extinction. Anti-predator sensitivity may exclude oscillation through homoclinic bifurcation and avert the prey population for any possible random extinction. Anti-predator sensitivity also decreases the predator population density and produces bistable dynamics. Higher values of anti-predator sensitivity may lead to the extinction of the predator population and benefit adult preys to persist with large population density. Below a threshold value of anti-predator sensitivity, it may possible to retain the predator population in the system by increasing the fear level of the predator. We also observe our fear-induced stage-structured model exhibits interesting and rich dynamical behaviors, various types of bistabilities in different bi-parameter planes. Finally, we discuss the potential impact of our findings.

Bifurcations, chaos, and multistability in a nonautonomous predator-prey model with fear.

Classical predator-prey models usually emphasize direct predation as the primary means of interaction between predators and prey. However, several field studies and experiments suggest that the mere presence of predators nearby can reduce prey density by forcing them to adopt costly defensive strategies. Adoption of such kind would cause a substantial change in prey demography. The present paper investigates a predator-prey model in which the predator's consumption rate (described by a functional response) is affected by both prey and predator densities. Perceived fear of predators leads to a drop in prey's birth rate. We also consider both constant and time-varying (seasonal) forms of prey's birth rate and investigate the model system's respective autonomous and nonautonomous implementations. Our analytical studies include finding conditions for the local stability of equilibrium points, the existence, direction of Hopf bifurcation, etc. Numerical illustrations include bifurcation diagrams assisted by phase portraits, construction of isospike and Lyapunov exponent diagrams in bi-parametric space that reveal the rich and complex dynamics embedded in the system. We observe different organized periodic structures within the chaotic regime, multistability between multiple pairs of coexisting attractors with intriguing basins of attractions. Our results show that even relatively slight changes in system parameters, perturbations, or environmental fluctuations may have drastic consequences on population oscillations. Our observations indicate that the fear effect alters the system dynamics significantly and drives an otherwise irregular system toward regularity.

Feature Selection Using a Neural Network With Group Lasso Regularization and Controlled Redundancy.

We propose a neural network-based feature selection (FS) scheme that can control the level of redundancy in the selected features by integrating two penalties into a single objective function. The Group Lasso penalty aims to produce sparsity in features in a grouped manner. The redundancy-control penalty, which is defined based on a measure of dependence among features, is utilized to control the level of redundancy among the selected features. Both the penalty terms involve the L2,1 -norm of weight matrix between the input and hidden layers. These penalty terms are nonsmooth at the origin, and hence, one simple but efficient smoothing technique is employed to overcome this issue. The monotonicity and convergence of the proposed algorithm are specified and proved under suitable assumptions. Then, extensive experiments are conducted on both artificial and real data sets. Empirical results explicitly demonstrate the ability of the proposed FS scheme and its effectiveness in controlling redundancy. The empirical simulations are observed to be consistent with the theoretical results.

Cooperation delay induced chaos in an ecological system.

In the present paper, we investigate the impact of time delay during cooperative hunting in a predator-prey model. We consider that cooperative predators do not aggregate in a group instantly, but individuals use different stages and strategies such as tactile, visual, vocal cues, or a suitable combination of these to communicate with each other. We observe that delay in hunting cooperation has stabilizing as well as destabilizing effects in the system. Also, for an increase in the strength of the delay, the system dynamics switch multiple times and eventually become chaotic. We see that depending on the threshold of time delay, the system may restore its original state or may go far away from its original state and unable to recollect its memory. Furthermore, we explore the dynamics of the system in different bi-parameter spaces and observe that for a particular range of other parameter values, the system dynamics switch multiple times with an increase of delay in all the planes. Different kinds of multistability behaviors, the coexistence of multiple attractors, and interesting changes in the basins of attraction of the system are also observed. We infer that depending on the initial population size and the strength of cooperation delay, the populations can exhibit stable coexistence, oscillating coexistence, or extinction of the predator species.

In Search of Trustworthy and Transparent Intelligent Systems With Human-Like Cognitive and Reasoning Capabilities.

At present we are witnessing a tremendous interest in Artificial Intelligence (AI), particularly in Deep Learning (DL)/Deep Neural Networks (DNNs). One of the reasons appears to be the unmatched performance achieved by such systems. This has resulted in an enormous hope on such techniques and often these are viewed as all-cure solutions. But most of these systems cannot explain why a particular decision is made (black box) and sometimes miserably fail in cases where other systems would not. Consequently, in critical applications such as healthcare and defense practitioners do not like to trust such systems. Although an AI system is often designed taking inspiration from the brain, there is not much attempt to exploit cues from the brain in true sense. In our opinion, to realize intelligent systems with human like reasoning ability, we need to exploit knowledge from the brain science. Here we discuss a few findings in brain science that may help designing intelligent systems. We explain the relevance of transparency, explainability, learning from a few examples, and the trustworthiness of an AI system. We also discuss a few ways that may help to achieve these attributes in a learning system.

Case report describing late balloon dilatation as a treatment for paravalvular leak with intuity elite rapid deployment valve.

Use of sutureless bioprostheses for aortic valve replacement has increased in recent years as compared to conventional prostheses, though with the potential issue of paravalvular leak, which requires close follow-up. We present this case report describing the successful treatment of paravalvular leak in a 65 year old man, who had NYHA class III symptoms post implantation of a 21 mm Intuity Elite rapid deployment bioprosthesis (Edwards Lifesciences, Irvine, CA). Diagnosis was established using TTE, TOE, and Cardiac MRI. Performing balloon dilatation using an Atlas Gold balloon (BARD Peripheral Vascular Inc., Tempe, AZ) treated the likely inadequate expansion of the subvalvular stent, leading to significant reduction in the paravalvular leak. At one month follow-up patient reported complete resolution of his symptoms. Successful percutaneous treatment of paravalvular leak following implantation of rapid deployment sutureless bioprosthesis provides a new treatment strategy for these patients; this strategy requires further validation.

Learning Optimized Structure of Neural Networks by Hidden Node Pruning With L1 Regularization.

We propose three different methods to determine the optimal number of hidden nodes based on L1 regularization for a multilayer perceptron network. The first two methods, respectively, use a set of multiplier functions and multipliers for the hidden-layer nodes and implement the L1 regularization on those, while the third method equipped with the same multipliers uses a smoothing approximation of the L1 regularization. Each of these methods begins with a given number of hidden nodes, then the network is trained to obtain an optimal architecture discarding redundant hidden nodes using the multiplier functions or multipliers. A simple and generic method, namely, the matrix-based convergence proving method (MCPM), is introduced to prove the weak and strong convergence of the presented smoothing algorithms. The performance of the three pruning methods has been tested on 11 different classification datasets. The results demonstrate the efficient pruning abilities and competitive generalization by the proposed methods. The theoretical results are also validated by the results.

Fear effect in prey and hunting cooperation among predators in a Leslie-Gower model.

The predation strategy for predators and the avoidance strategy of prey are important topics in ecology and evolutionary biology. Both prey and predators adjust their behaviours in order to gain the maximal benefits and to increase their biomass for each. In the present paper, we consider a modified Leslie-Gower predator-prey model where predators cooperate during hunting and due to fear of predation risk, prey populations show anti-predator behaviour. We investigate step by step the impact of hunting cooperation and fear effect on the dynamics of the system. We observe that in the absence of fear effect, hunting cooperation can induce both supercritical and subcritical Hopf- bifurcations. It is also observed that fear factor can stabilize the predator-prey system by excluding the existence of periodic solutions and makes the system more robust compared to hunting cooperation. Moreover, the system shows two different types of bi-stabilities behaviour: one is between coexisting equilibrium and limit cycle oscillation, and another is between prey-free equilibrium and coexisting equilibrium. We also observe generalized Hopf-bifurcation and Bogdanov-Takens bifurcation in two parameter bifurcation analysis. We perform extensive numerical simulations for supporting evidence of our analytical findings.

Contemporary Management of Stent Failure: Part One.

The occurrence of in-stent restenosis (ISR) still remains a daunting challenge in the current era, despite advancements in coronary intervention technology. The authors explore the underlying pathophysiology and mechanisms behind ISR, and describe how the use of different diagnostic tools helps to best elucidate these. They propose a simplistic algorithm to manage ISR, including a focus on how treatment strategies should be selected and a description of the contemporary technologies available. This article aims to provide a comprehensive outline of ISR that can be translated into evidence-based routine clinical practice, with the aim of providing the best outcomes for patients.

Weight Noise Injection-Based MLPs With Group Lasso Penalty: Asymptotic Convergence and Application to Node Pruning.

The application and theoretical analysis of fault tolerant learning are very important for neural networks. Our objective here is to realize fault tolerant sparse multilayer perceptron (MLP) networks. The stochastic gradient descent method has been employed to perform online learning for MLPs. For weight noise injection-based network models, it is a common strategy to add a weight decay regularizer while constructing the objective function for learning. However, this l2 -norm penalty does not generate sparse optimal solutions. In this paper, a group lasso penalty term is used as a regularizer, where a group is defined by the set of weights connected to a node from nodes in the preceding layer. Group lasso penalty enables us to prune redundant hidden nodes. Due to its nondifferentiability at the origin, a smooth approximation of the group lasso penalty is developed. Then, a rigorous proof for the asymptotic convergence of the learning algorithm is provided. Finally, some simulations have been performed to verify the sparseness of the network and the theoretical results.

Distributed Adaptive Tracking Synchronization for Coupled Reaction-Diffusion Neural Network.

This paper considers the tracking synchronization problem for a class of coupled reaction-diffusion neural networks (CRDNNs) with undirected topology. For the case where the tracking trajectory has identical individual dynamic as that of the network nodes, the edge-based and vertex-based adaptive strategies on coupling strengths as well as adaptive controllers, which demand merely the local neighbor information, are proposed to synchronize the CRDNNs to the tracking trajectory. To reduce the control costs, an adaptive pinning control technique is employed. For the case where the tracking trajectory has different individual dynamic from that of the network nodes, the vertex-based adaptive strategy is proposed to drive the synchronization error to a relatively small area, which is adjustable according to the parameters of the adaptive strategy. This kind of adaptive design can enhance the robustness of the network against the external disturbance posed on the tracking trajectory. The obtained theoretical results are verified by two representative examples.